Speech Technology in (Re)Habilitation of Persons with Communication Disabilities

When people engage in conversation, they adapt their speaking style of their conversational partner in a number of ways. They have been shown to adopt their interlocutor's way of describing objects and to align their accent, their syntax, their pitch range, and their speaking rate to their partner's ‐‐ as well as their gestures. They also adapt their turn‐taking style and use of cue phrases to match their partner's. These types of entrainment have been shown to correlate with various measures of task success and dialogue naturalness. While there is considerable evidence for lexical entrainment from laboratory experiments, less is known about other types of acoustic‐prosodic and discourse‐ level entrainment and little work has been done to examine entrainments in multiple modalities for the same dialogue. I will discuss research in entrainment in multiple dimensions on the Columbia Games Corpus and the Switchboard Corpus. Our goal is to understand how the different varieties of entrainment correlate with one another and to determine which types of entrainment will be both useful and feasible to model in Spoken Dialogue Systems.. Abstract How does the human brain represent meanings of words and pictures in terms of neural activity? This talk will present our research addressing this question, by applying machine learning algorithms to fMRI and MEG brain image data. One line of our research involves training classifiers that identify which word a person is thinking about, based on their observed neural activity. A second line involves training computational models that predict the neural activity associated with arbitrary English words, including words for which we do not yet have brain image data. A third line of work involves examining neural activity at millisecond time resolution during the comprehension of words and phrases.